Resource allocation and congestion control in vehicular ad-hoc networks through optimization algorithms and artificial intelligence

[SPA] Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. En los últimos años la creciente demanda de la industria del transporte junto con requisitos de seguridad cada vez más estrictos han promovido el rápido desarrollo de las comunicaciones vehiculares. Tales comunicaciones se basan en el intercambio de mensajes periódicos (beacons) que contienen información crítica de los vehículos. Esta difusión de información da origen a lo que comúnmente se denomina conciencia cooperativa, que permite ampliar las capacidades de numerosos sistemas de asistencia en carretera y las diferentes aplicaciones de seguridad. Ciertamente, la difusión de información entre vehículos es la base de la conducción autónoma y reduce drásticamente el riesgo de colisión y otros eventos indeseados. Sin embargo, es importante tener en cuenta que la carga agregada de los beacons transmitidos puede congestionar rápidamente el canal, comprometiendo la recepción de paquetes y, por lo tanto, poniendo en peligro las ventajas que ofrecen tales comunicaciones. Para garantizar la disponibilidad del canal tanto para la recepción correcta de mensajes de emergencia y de las mínimas balizas necesarias para satisfacer los requisitos de las aplicaciones de seguridad, una determinada fracción del canal debe de ser reservada. En la literatura relacionada, el control de la congestión se ha abordado mediante el ajuste de varios parámetros de transmisión (tasa de mensaje, potencia y tasa de bit), pero todavía existen numerosos desafíos por abordar. Por ejemplo, aunque los parámetros de transmisión suelen ajustarse individualmente debido a la simplicidad del problema de optimización, aquí se muestran las ventajas de ajustar varios parámetros de forma simultánea. En esta tesis, se propone el uso de diferentes algoritmos distribuidos que alcancen el nivel de congestión deseado sin requerir infraestructura ninguna en carretera. La primera parte de esta tesis aborda la asignación de la tasa de balizamiento mediante la maximización de la utilidad de red (NUM) y diferentes métricas de riesgo como el tiempo de colisión y la velocidad de la carretera de aviso. En la segunda parte, no solo se estudian diferentes combinaciones consistentes de parámetros, sino que también nos sumergimos en el paradigma de los algoritmos no cooperativos, en los que no se requiere información de los vehículos vecinos. El problema de control de la congestión es formulado como un Proceso de Decisión de Markov (MDP) y resuelto mediante técnicas de inteligencia artificial, más concretamente, mediante aprendizaje por refuerzo (RL). Se proponen diferentes soluciones que van desde simples métodos tabulares, adecuados para entornos discretos, como Q-learning, hasta funciones de aproximación más complejas adecuadas para espacios continuos, como SARSA basado en semi-gradiente o redes neuronales artificiales.[ENG] This doctoral dissertation has been presented in the form of thesis by publication. The ever-increasing growth of the transportation industry demands combined with new safety requirements has triggered the development of vehicular communications. These communications among vehicles are based on the exchange of periodical messages or beacons containing valuable information about vehicle state. This gives rise to the socalled cooperative awareness, which allows extending the capabilities of numerous driver assistance systems and safety applications. Disseminating information among vehicles certainly lessens the risk of collision and other undesired events. Nevertheless, the aggregated beaconing load can rapidly jam the channel, compromising packet reception, and therefore endangering the advantages offered by such communications. To guarantee the availability of the channel for emergency messages and the minimum beacons receptions that satisfy safety application requirements, a given fraction of the channel capacity should be available. This congestion control has been addressed by adjusting several transmission parameters but some challenges are still unresolved. Although these parameters are usually optimized individually because of the convexity of the optimization problem, we show the advantages of combining them. In this thesis, we propose the use of different distributed algorithms that reach the desired congestion level without explicitly requiring any costly infrastructure. The first part of this thesis addresses beaconing rate allocation. We propose several distributed solutions based on Network Utility Maximization (NUM) and different risk metrics such as time-to-collision and advisory road speed. In the second part, we not only study different combinations of well-coupled parameters but also dive into the paradigm of noncooperative algorithms, in which no information from neighboring vehicles or centralized infrastructure are required. We formulate the congestion control problem as a Markov Decision Process and solve it by means of different reinforcement learning techniques. In particular, we propose different solutions ranging from tabular methods suitable for simple and discrete environments, like Q-learning, to more complex functions approximations for continuous action-state spaces, such as Semi-gradient SARSA or artificial neural networks.Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. Está formada por estos seis artículos: 1. (j1) Aznar-Poveda, J., Egea-Lopez, E., Garcia-Sanchez, A. J., and Pavon-Mariño, P. (2019, October). Time-to-Collision-Based Awareness and Congestion Control for Vehicular Communications. IEEE Access, 7, 154192-154208. DOI: 10.1109/ACCESS.2019.2949131. 2. (c1) Aznar-Poveda, J., Egea-Lopez, E., and Garcia-Sanchez, A. J. (2020, May). Cooperative Awareness Message Dissemination in EN 302 637-2: An Adaptation for Winding Roads. IEEE 91st Vehicular Technology Conference (VTC2020-Spring) (pp. 1-5). IEEE. DOI: 10.1109/VTC2020-Spring48590.2020.9128815. 3. (c2) Aznar-Poveda, J., Egea-Lopez, E., Garcia-Sanchez, A. J., and Garcia-Haro, J. (2020, July). Advisory Speed Estimation for an Improved V2X Communications Awareness in Winding Roads. In 2020 22nd International Conference on Transparent Optical Networks (ICTON) (pp. 1-4). IEEE. DOI: 10.1109/ICTON51198.2020.9203478 4. (j2) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, January). MDPRP: A Q-Learning Approach for the Joint Control of Beaconing Rate and Transmission Power in VANETs. IEEE Access, 9, 10166-10178. DOI: 10.1109/ACCESS.2021.3050625 5. (j3) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, August). Simultaneous Data Rate and Transmission Power Adaptation in V2V Communications: A Deep Reinforcement Learning Approach. IEEE Access 9, 122067-122081. DOI: 10.1109/ACCESS.2021.3109422 6. (j4) Aznar-Poveda, J., Garcia-Sanchez, A. J., Egea-Lopez, E., and Garcia-Haro, J. (2021, December). Approximate Reinforcement Learning to Control Beaconing Congestion in Distributed Networks. Scientific Reports, 12, 142. DOI: 10.1038/s41598-021-04123-9Escuela Internacional de Doctorado de la Universidad Politécnica de CartagenaUniversidad Politécnica de CartagenaPrograma de Doctorado en Tecnologías de la Información y las Comunicacione

Simultaneous data rate and transmission power adaptation in V2V communications: A deep reinforcement learning approach

In Vehicle-to-Vehicle (V2V) communications, channel load is key to ensuring the appropriate operation of safety applications as well as driver-assistance systems. As the number of vehicles increases, so do their communication messages. Therefore, channel congestion may arise, negatively impacting channel performance. Through suitable adjustment of the data rate, this problem would be mitigated. However, this usually involves using different modulation schemes, which can jeopardize the robustness of the solution due to unfavorable channel conditions. To date, little effort has been made to adjust the data rate, alone or together with other parameters, and its effects on the aforementioned sensitive safety applications remain to be investigated. In this paper, we employ an analytical model which balances the data rate and transmission power in a non-cooperative scheme. In particular, we train a Deep Neural Network (DNN) to precisely optimize both parameters for each vehicle without using additional information from neighbors, and without requiring any additional infrastructure to be deployed on the road. The results obtained reveal that our approach, called NNDP, not only alleviates congestion, leaving a certain fraction of the channel available for emergency-related messages, but also provides enough transmission power to fulfill the application layer requirements at a given coverage distance. Finally, NNDP is thoroughly tested and evaluated in three realistic scenarios and under different channel conditions, demonstrating its robustness and excellent performance in comparison with other solutions found in the scientific literature.This work was supported in part by the AEI/FEDER/UE [Agencia Estatal de Investigación (AEI), Fondo Europeo de Desarrollo Regional (FEDER), and Unión Europea (UE)] under Grant PID2020-116329GB-C22 [ARISE2: Future IoT Networks and Nano-networks (FINe)] and Grant PID2020-112675RB-C41 (ONOFRE-3), in part by the Fundación Séneca, Región de Murcia, under Grant 20889/PI/18 (ATENTO), and in part by the LIFE project (Fondo SUPERA COVID-19 through the Agencia Estatal Consejo Superior de Investigaciones Científicas CSIC, Universidades Españolas, and Banco Santander). The work of Juan Aznar-Poveda was supported by the Spanish Ministerio de Educación, Cultura y Deporte (MECD) through the Formación de Personal Investigador (FPI) Predoctoral Scholarship under Grant BES-2017-08106

MDPRP: A Q-learning approach for the joint control of beaconing rate and transmission power in VANETs

Vehicular ad-hoc communications rely on periodic broadcast beacons as the basis for most of their safety applications, allowing vehicles to be aware of their surroundings. However, an excessive beaconing load might compromise the proper operation of these crucial applications, especially regarding the exchange of emergency messages. Therefore, congestion control can play an important role. In this article, we propose joint beaconing rate and transmission power control based on policy evaluation. To this end, a Markov Decision Process (MDP) is modeled by making a set of reasonable simplifying assumptions which are resolved using Q-learning techniques. This MDP characterization, denoted as MDPRP (indicating Rate and Power), leverages the trade-off between beaconing rate and transmission power allocation. Moreover, MDPRP operates in a non-cooperative and distributed fashion, without requiring additional information from neighbors, which makes it suitable for use in infrastructureless (ad-hoc) networks. The results obtained reveal that MDPRP not only balances the channel load successfully but also provides positive outcomes in terms of packet delivery ratio. Finally, the robustness of the solution is shown since the algorithm works well even in those cases where none of the assumptions made to derive the MDP model apply.This work was supported in part by the AIM Project [Agencia Estatal de Investigación (AEI)/Fondo Europeo de Desarrollo Regional (FEDER), Unión Europea (UE)] under Grant TEC2016-76465-C2-1-R, in part by the Fundación Séneca, Región de Murcia, through the ATENTO Project, under Grant 20889/PI/18, and in part by the LIFE (Fondo SUPERA Covid-19 funded by the Agencia Estatal Consejo Superior de Investigaciones Científicas CSIC, Universidades Españolas, and Banco Santander). The work of Juan Aznar-Poveda was supported by the Spanish Ministerio de Educación, Cultura y Deporte (MECD) for the FPI Grant BES-2017-081061

Approximate reinforcement learning to control beaconing congestion in distributed networks

In vehicular communications, the increase of the channel load caused by excessive periodical messages (beacons) is an important aspect which must be controlled to ensure the appropriate operation of safety applications and driver-assistance systems. To date, the majority of congestion control solutions involve including additional information in the payload of the messages transmitted, which may jeopardize the appropriate operation of these control solutions when channel conditions are unfavorable, provoking packet losses. This study exploits the advantages of non-cooperative, distributed beaconing allocation, in which vehicles operate independently without requiring any costly road infrastructure. In particular, we formulate the beaconing rate control problem as a Markov Decision Process and solve it using approximate reinforcement learning to carry out optimal actions. Results obtained were compared with other traditional solutions, revealing that our approach, called SSFA, is able to keep a certain fraction of the channel capacity available, which guarantees the delivery of emergency-related notifications with faster convergence than other proposals. Moreover, good performance was obtained in terms of packet delivery and collision ratios.This research has been supported by the projects AIM, ref. TEC2016-76465-C2-1-R, ARISE2 “Future IoT Networks and Nano-networks (FINe)” ref. PID2020-116329GB-C22, ONOFRE-3, ref. PID2020-112675RB-C41 [Agencia Estatal de Investigación (AEI), European Regional Development Fund (FEDER), European Union (EU)], ATENTO, ref. 20889/PI/18 (Fundación Séneca, Región de Murcia), and LIFE [Fondo SUPERA Covid-19, funded by Agencia Estatal Consejo Superior de Investigaciones Científicas (CSIC), Universidades Españolas and Banco Santander]. J.A.P. thanks the Spanish MECD for an FPI grant ref. BES-2017-081061. Finally, the authors acknowledge Laura Wettersten for her contribution in reviewing the grammar and spell of the manuscript

Dispositivo y método para la medición de ácido ascórbico y antocianos en zumos

Número de publicación: 2 728 448 Número de solicitud: 201930555Dispositivo y método para la medición de ácido ascórbico y antocianos en zumos. Un dispositivo y un método para la medición de ácidoascórbico y antocianos en zumos que comprende un sensor (1) amperométrico que comprende un primer electrodo de trabajo (3) y un segundo electrodo de trabajo (4); unos medios electrónicos de captura y procesado de datos (6, 7, 8, 9); una interfaz gráfica de gestión y/o configuración (10); y donde el primer electrodo de trabajo (3) no está tratado con ascorbatooxidasa y un segundo electrodo de trabajo (4) que está tratado con ascorbato oxidasa; y donde los medios electrónicos (6, 7, 8, 9) están configurados para estimular el sensor (1) según un rango de tensiones predefinido en función del tipo de zumo seleccionado por un usuario en la interfaz gráfica de gestión y/o configuración (10), y obtener simultáneamente la lectura de corriente debida a la oxidación del zumo estudiado.Universidad Politécnica de Cartagen

Cooperative awareness message dissemination in EN 302 637-2: an adaptation for winding roads

This paper evaluates the performance of the Cooperative Awareness Message (CAM) dissemination stated in the European Standard EN 302 637-2 in risky sharp bends and winding roads. We propose a novel triggering condition based on the dynamic estimation of the road radius, used as a risk metric. So as lower the radius, the higher the beaconing rate. As a case study, two real road sections, with different lengths and angles, have been simulated to prove that both a better awareness and responsiveness are achieved in the vehicles, to later ensure a proper application layer functioning. Finally, congestion constraint is also tested to check that no significant interferences are found in the described behavior.This work has been supported by the project AIM, ref. TEC2016-76465-C2-1-R (AEI/FEDER, UE). In particular, Juan Aznar-Poveda thanks the Spanish MECD for an FPI predoctoral fellowship (ref. BES-2017-081061)

Time-to-collision-based awareness and congestion control for vehicular communications

Vehicular wireless communications require both congestion control to guarantee the availability of a fraction of the bandwidth for safety-related event-driven messages in emergency cases, and awareness control to adapt the beaconing activity to the application needs and surrounding traf_c situation. Most current approaches either ignore the traf_c situation and only adapt the beaconing rate to the channel congestion state or override the congestion control limits, leading to questionable results in both cases. In this paper, we conceive and validate a novel approach, combining both aspects. Based on distributed Network Utility Maximization (NUM), our algorithm satis_es the constraints on channel availability, whereas the safety of the surrounding traf_c situation is captured with a time-to-collision metric, used to assign priorities in the optimal allocation problem. The performance of the proposed approach is validated and compared to other popular algorithms. Results show that our proposal automatically anticipates a potential increase in rate due to a critical safety situation, but does not interfere with the reserved bandwidth for safety applications.This work was supported in part by Spanish MECD/AEI/FEDER under Grant TEC2016-76465-C2-1-R, in part by Spanish MINECO/AEI/FEDER under Grant TEC2017-84423-C3-2-P (ONOFRE-2), in part by e-DIVITA under Grant 20509/PDC/18 (Proof of Concept, 2018), and in part by ATENTO Projects under Grant 20889/PI/18 (Fundación Seneca, Región de Murcia). The work of J. Aznar-Poveda was supported by the MECD under Grant BES-2017-081061

Advisory speed estimation for an improved V2X communications awareness in winding roads

The European Telecommunications Standards Institute (ETSI) establishes a Decentralized Congestion Control (DCC) which triggers the so-called Cooperative Awareness Message (CAM) depending on vehicle kinematics. However, this algorithm hence called CAM-DCC represents a challenge in the triggering rules of these messages. In particular, it lacks (i) awareness of the neighboring vehicles and (ii) efficient use of the channel bandwidth. Consequently, information gaps related to the road environment might give rise to non-compliances in the application layer requirements of the vehicles, which could potentially threaten the drivers’ safety, most particularly in hazardous roads. To overcome these flaws, we first study the CAM generation trigger focused on the vehicle heading in risky curves or winding roads. Then, we evaluate both scenarios tuning different triggering thresholds and including additional mechanisms such as the comparison of the current speed respect to the estimated advisory speed over time. Different computer simulations have been conducted in two real road sections to validate our proposal. Results reveal significant better performance in terms of awareness and channel usage.This work has been supported by the by the projects AIM, ref. TEC2016-76465-C2-1-R (AEI/FEDER, UE), and ATENTO, ref. 20889/PI/18 (Fundación Séneca, Región de Murcia). In particular, Juan Aznar-Poveda thanks the Spanish MECD for an FPI predoctoral fellowship (ref. BES-2017-081061)

Quick and Cost-Effective Estimation of Vitamin C in Multifruit Juices Using Voltammetric Methods

Ascorbic Acid (AA) is a natural and powerful water-soluble antioxidant associated with long-lasting food products. As time passes, the AA content in products sharply decreases, and they become increasingly degraded. There are several techniques to precisely quantify AA concentrations. However, most of them employ costly laboratory instruments, such as High-Performance Liquid Chromatography (HPLC) or complex electrochemical methods, which make unfeasible recurrent AA measurements along the entire supply chain. To address this issue, we contribute with an in-field and real-time voltammetric method, carried out with a low-cost, easy-to-use, and portable device. An unmodified Screen-Printed Electrode (SPE) is used together with the device to achieve short reading times. Our method has been extensively tested in two multifruit juices using three different SPEs. Calibration curves and Limit of Detection were derived for each SPE. Furthermore, periodic experiments were conducted to study the shelf life of juices under consideration. During the analysis, a set of assays for each SPE were implemented to determine the remaining AA amount per juice and compare it with that obtained using HPLC under the same conditions. Results revealed that our cost-effective device is fully comparable to the HPLC equipment, as long as the juice does not include certain interferents; a scenario also contemplated in this article